Skip to content

[ef-30] feat: post-publish smoke test pipeline + bump to 0.0.1-beta.6#27

Merged
NiveditJain merged 1 commit into
mainfrom
ef-30
Apr 7, 2026
Merged

[ef-30] feat: post-publish smoke test pipeline + bump to 0.0.1-beta.6#27
NiveditJain merged 1 commit into
mainfrom
ef-30

Conversation

@NiveditJain

@NiveditJain NiveditJain commented Apr 7, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Member

Summary

  • Adds .github/workflows/smoke-test.yml that auto-triggers after every successful npm publish
  • Tests on ubuntu, macos, and windows in parallel
  • Exponential backoff npm registry check: immediate → 2s → 4s → 8s → 30s → 60s
  • Smoke tests: --version output matches released version, /policies and /projects HTML matches golden snapshots
  • HTML comparison normalizes away Next.js build IDs and chunk hashes so cross-build diffs don't produce false failures
  • Bumps version to 0.0.1-beta.6

Golden snapshots

The HTML snapshot files under .github/smoke-test/expected/ are currently placeholders. To enable the HTML comparison step, run locally and commit:

CLAUDE_PROJECTS_PATH=/tmp/empty-projects FAILPROOFAI_TELEMETRY_DISABLED=1 failproofai &
curl -s http://localhost:8020/policies > .github/smoke-test/expected/policies.html
curl -s http://localhost:8020/projects > .github/smoke-test/expected/projects.html

Until then, the workflow skips the comparison and prints a warning — all other smoke tests (install, version) still run.

Test plan

  • Merge and publish a release → confirm "Post-Publish Smoke Test" workflow triggers automatically
  • gh workflow run smoke-test.yml to manually trigger against the current published version
  • Generate and commit golden snapshots, then re-run to confirm HTML comparison passes

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Introduced automated post-publish smoke testing workflow that validates package installation and verifies server functionality across Linux, macOS, and Windows platforms after each npm release.
    • Updated version to 0.0.1-beta.6.

@NiveditJain @claude
[ef-30] feat: post-publish smoke test pipeline + bump to 0.0.1-beta.6
8d75db0 
Adds .github/workflows/smoke-test.yml that triggers after every npm
publish, installs the package on ubuntu/macos/windows with exponential
backoff (0→2→4→8→30→60s) while waiting for registry propagation, then
verifies --version and HTML output of /policies and /projects pages.

Golden HTML snapshots in .github/smoke-test/expected/ are placeholders
until generated locally and committed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 7, 2026 •
edited
Loading

Copy link
Copy Markdown
ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: 2720cfb2-fecc-4d8a-92e1-09bd35f57dd7

📥 Commits

Reviewing files that changed from the base of the PR and between e8e10fe and 8d75db0.

📒 Files selected for processing (4)
  • .github/smoke-test/expected/policies.html
  • .github/smoke-test/expected/projects.html
  • .github/workflows/smoke-test.yml
  • package.json
📝 Walkthrough

Walkthrough

This PR introduces a post-publish smoke test workflow that validates the npm package functions correctly across multiple operating systems. It includes placeholder snapshot files for HTML comparison and a GitHub Actions workflow that installs the published package, runs the application server, captures endpoint responses, and compares them against expected snapshots.

Changes

Cohort / File(s) Summary
Smoke Test Snapshots
.github/smoke-test/expected/policies.html, .github/smoke-test/expected/projects.html		 Placeholder HTML files containing instructions for generating actual endpoint snapshots by running the app locally and fetching /policies and /projects routes. Workflow will skip comparison until real snapshots are committed.
Smoke Test Workflow
.github/workflows/smoke-test.yml		 New GitHub Actions workflow triggered on successful npm publish or manual dispatch. Installs the published package globally, starts the server, fetches HTML responses from /policies and /projects endpoints across OS matrix (Ubuntu, macOS, Windows), normalizes dynamic content (build hashes, RSC scripts), and compares against expected snapshots. Uploads captured HTML as artifacts on failure.
Version Bump
package.json		 Increments package version from 0.0.1-beta.5 to 0.0.1-beta.6.

Sequence Diagram(s)

sequenceDiagram
    actor GitHub as GitHub Actions
    participant npm as npm Registry
    participant app as failproofai App
    participant http as HTTP Endpoints
    participant python as Python Validator
    
    GitHub->>npm: Resolve package version
    npm-->>GitHub: Return version
    GitHub->>npm: Poll for version availability
    npm-->>GitHub: Version available
    GitHub->>GitHub: npm install -g failproofai
    GitHub->>app: Start server (CLAUDE_PROJECTS_PATH set)
    app-->>GitHub: Server ready (PID recorded)
    GitHub->>http: Poll http://localhost:8020/
    http-->>GitHub: Server reachable
    GitHub->>http: Fetch /policies endpoint
    http-->>GitHub: HTML response
    GitHub->>http: Fetch /projects endpoint
    http-->>GitHub: HTML response
    GitHub->>app: Stop server
    GitHub->>python: Run comparison script
    python->>python: Normalize dynamic content<br/>(strip __NEXT_DATA__, RSC scripts,<br/>build hashes)
    python-->>GitHub: Comparison result
    alt Mismatch Detected
        GitHub->>GitHub: Upload artifacts
    end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 A smoke test hops to life,
No more publish-time strife,
Across Linux, Mac, and Windows it flies,
Checking policies and projects with keen rabbit eyes,
Quality assured, our package now certified! 🚀

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@NiveditJain NiveditJain merged commit 3cce580 into main Apr 7, 2026
8 checks passed
NiveditJain added a commit that referenced this pull request Apr 7, 2026
@NiveditJain @claude
[ef-30] feat: post-publish smoke test pipeline + bump to 0.0.1-beta.6 (…
c03a8c2 
…#27)

Adds .github/workflows/smoke-test.yml that triggers after every npm
publish, installs the package on ubuntu/macos/windows with exponential
backoff (0→2→4→8→30→60s) while waiting for registry propagation, then
verifies --version and HTML output of /policies and /projects pages.

Golden HTML snapshots in .github/smoke-test/expected/ are placeholders
until generated locally and committed.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@NiveditJain NiveditJain deleted the ef-30 branch April 21, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@NiveditJain